Method for providing control commands for a vehicle steerable by means of an operating unit and control system for a steerable vehicle of this type

ABSTRACT

Method and control system for providing control commands for a vehicle steerable by an operating unit or for an image acquisition device of the vehicle by sequential display of acquired image data on an image display device of the operating unit connected to the vehicle via a data transfer connection. The method includes acquiring data of an image recorded by the image acquisition device, acquiring vehicle state data at a time the image was recorded, and compressing the image data. The method also includes transferring the compressed image data and the vehicle state data via the data transfer connection to the operating unit, decompressing the transferred image data to form a displayable image, and displaying the displayable image on the image display device. The method further includes entering an operator command into the operating unit to at least one of change a direction of movement of the vehicle and change a direction of view of the image acquisition device, calculating a new image from the displayable image that takes into account the transferred vehicle state data and the entered operator command, displaying the new image on the image display device instead of the displayable image, and transferring a control command from the operator command to the vehicle via the data transfer connection.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. §119 of GermanPatent Application No. 10 2009 042 172, filed on Sep. 18, 2009, thedisclosure of which is expressly incorporated by reference herein in itsentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for providing control commandsfor a vehicle steerable by an operating unit or for an image acquisitiondevice of the vehicle by sequential display of image data of the imageacquisition device of the vehicle on an image display device of theoperating device connected to the vehicle via a data transferconnection. It furthermore relates to a control system with at least onesteerable vehicle, at least one operating device for the vehicle and adata transfer device for the exchange of data and/or signals between thevehicle and the operating device. Preferably the vehicle is an aircraft,such as, for example, an armed missile.

2. Discussion of Background Information

If a vehicle that has an image acquisition device and transfers theacquired images to an operating unit on which an operator sees thedisplayed images is controlled by the operator based on these images, asignificant latency occurs between the acquisition of an image in thevehicle and the arrival of a control signal due to an operator commandcarried out on the basis of the acquired and displayed image. Thislatency can mean that operator commands as a reaction of the operator toa recorded image arrive at the vehicle with marked delay, so that thecontrol of the vehicle cannot be carried out successfully in thismanner.

To solve this latency problem, it is known with a missile to provide adata transfer device with a broadband data link between the missile andthe operating unit. The operator thereby receives the image sequence orvideo sequence recorded by the image acquisition device displayed on theimage display device of the operating unit with an imperceptible timedelay and thus can correct the target point, for example, during theflight of the missile. The image is transmitted to the operating unit ofthe ground station without almost any latency due to the rapid dataconnection. During the flight, the details of the target become visibleto the operator with increasing clarity. Moreover, due to the fast dataconnection, the images can be transmitted at a high image refresh rateso that a video sequence is produced virtually free from jolts, whichprovides the operator with a good basis for a target correction. Thedata transfer is thereby carried out via an optical fiber cable, whichis unwound from a reel in the missile during the flight. This opticalfiber cable thus represents a physical connection between the flyingmissile and the operating unit. It is obvious that a physical connectionof this type can be maintained only over a limited distance between theoperating unit and the target point. An alternative to this physicaldata transfer connection by optical fiber cable or copper cable can be abroadband radio connection, which, however, can be realized only withdirect sight connection between the operating unit and the missile andonly over a relatively short distance. If a direct sight connectionbetween the operating unit and the missile in flight is not possible andif cable cannot or should not be carried on board, the only possibilityis a transfer at much lower data rates via radio.

So-called “man-in-the-loop” controls of this type, in which an operatoroperates the vehicle by remote control based on images recorded in thevehicle, have two considerable advantages. During the approach by thevehicle to a target or a target region, the operator can orient himselfbased on the transmitted images and reconnoiter the region regarding atarget or alternative targets. To this end, the operator can pivot theimage recording device of the vehicle via the remote control, forexample, if the vehicle is underway for a sufficient length of time.Furthermore, the operator has the option of selecting the concretetarget and carrying out corresponding path corrections. The closer thevehicle approaches the target or the target region, the more preciselythe target can be identified. On the basis of this identification eithera more precise orientation in the target direction can be made or themission can be aborted.

If only a narrow-band data transfer connection is available instead of abroadband data transfer connection, the remote control of the vehicle bythe operator will become markedly more difficult. Even a simple pivotingof the seeker head with the image acquisition device by the user, forexample, carried out by a joystick on the operating unit, is virtuallyimpossible for the operator to handle without additional precautions.The response of the movement of the control lever, that is, the enteredoperator command for the seeker head, arrives at the operating unit as aswivel visible in the image only after a long delay and long after thecontrol lever has been actuated and is consequently displayed there onthe image display device with a long delay. These latency periods orreaction times can be in the range of seconds even at high datacompression rates. A target selection, that is, a control of the vehicleitself, cannot be carried out in this manner, either.

An alternative to this “man-in-the-loop” control affected by latencyproblems would be the use of tracking methods, i.e., an automatictracking of the target in the image. However, tracking methods of thistype require very sophisticated image processing devices on board thevehicle and substantially increase the cost of the vehicle.

SUMMARY OF THE INVENTION

Embodiments of the present invention are directed to a method forproviding control commands for a vehicle steerable by an operating unit,in particular, a missile, or for an image acquisition device of thevehicle by a sequential display of image data of the image acquisitiondevice of the vehicle on an image display device of the operating deviceconnected to the vehicle via a data transfer connection. The methodpermits the reliable transmission of control commands to the vehiclewhile minimizing the effects of latency even with narrow-band datatransfer connections.

Furthermore, embodiments are directed to a control system with at leastone steearable vehicle, at least one operating device for the vehicleand a data transfer device for the exchange of data between the vehicleand the operating device, which, even with a narrow-band data transferconnection, renders possible a reliable remote control of the vehiclebased on the images transferred to the operating device.

According to embodiments, a method according to the invention caninclude the following:

-   -   a) Acquisition of the data of an image recorded by the image        acquisition device;    -   b) Acquisition of vehicle state data at the time the image was        taken in step a);    -   c) Compression of the image data acquired in step a);    -   d) Transfer of the image data compressed in step c) together        with the vehicle state data acquired in step b) via the data        transfer connection to the operating unit;    -   e) Decompression of the transferred image data to form an image;    -   f) Display of the image obtained in step e) on the image display        device;    -   g) Repetition of steps a) through f).

With the entry of an operator command into the operating unit, whichcauses a change of the direction of movement of the vehicle and/or achange in the direction of view of the image acquisition device, thefollowing additional steps may also be carried out according to theinvention:

-   -   h) From the last image obtained in step e) a new image is        calculated by extrapolation of the previous image, taking into        account the vehicle state data transferred in step d) and the        operator command entered and    -   i) Displayed on the image display device instead of the image        obtained in step e) and    -   j) A control command determined from the operator command is        transferred to the vehicle via the data transfer connection.

Through the method according to embodiments of the invention, the imagedisplay device does not display to the operator the real images arrivingwith delay from the image acquisition device, but high-resolutionhypothetical images that are produced based on a forecast. Ahypothetical image of this type is based on a real recorded originalimage and is converted to a hypothetical image based on the knownvehicle state data and the entered operator command. Consequently, animage is displayed which prospectively corresponds to the current viewof the image acquisition device.

This so-called prediction method carried out in the operating unit ofthe ground station and the use of highly compressing image compressionmethods with low image refresh rates make it possible for the user tocarry out a pivoting of the seeker head and/or a secure target selectionas in real-time viewing of the scene recorded by the image acquisitiondevice. In this manner, the latency that exists in reality has virtuallyno impact on the stability of the steering control circuit. Since noimage processing methods are required on the vehicle apart from the datacompression for image transfer, the method according to the inventionleads to a very cost-effective method for the remote control of thevehicle. The prerequisite for this is only that navigation informationis available on the vehicle.

A preferred further development of the method is characterized in thatthe vehicle state data contain the navigated position of the vehicle,the location at least of the longitudinal axis of the vehicle and itsalignment as well as the target position to be targeted by the vehicle.

It is furthermore advantageous if the vehicle state data contain data onthe direction of the optical axis of the image acquisition device and ofthe view direction thereof.

It is particularly advantageous thereby if the data of the direction ofthe optical axis, preferably as Eulerian angles, quaternions ortransform matrixes are recorded and transferred. If an image acquisitiondevice that is pivotable on the vehicle is available, it is advantageousto record and transfer the measured pivot angle (gimbal angle).

In a further preferred embodiment of the method according to theinvention, the direction to the target position and/or the direction ofthe optical axis and/or the direction of the longitudinal axis of thevehicle is displayed in the image, preferably as cross hairs.

In a particularly preferred further development of the method accordingto embodiments of the invention, the extrapolation of the previous imagewith the entry of an operator command to change the viewing angle of theimage acquisition device can include the following:

-   h1) Determining the direction of the optical axis of the image    acquisition device from the vehicle state data transmitted in step    d);-   h2) Determining the commanded direction of the optical axis from the    entered operator command;-   h3) Determining a new image by changing the previous image taking    into account the commanded direction of the optical axis;-   h4) Transmitting the commanded direction of the optical axis    acquired in step h2) to the vehicle;-   h5) Calculating new commanded gimbal angles for the image    acquisition device in the onboard computer of the vehicle using, the    transmitted commanded direction of the optical axis and the    navigated vehicle location and-   h6) Pivoting the image acquisition device according to the commanded    gimbal angles calculated in step h5).

Further, it can be particularly advantageous if the following steps arecarried out to determine the new image in the above-described step h3):

-   h3.1) Estimating a new direction of the optical axis starting from    the previous direction of the optical axis based on a model of the    vehicle dynamics and the gimbal dynamics of the pivotable image    acquisition device using the commanded direction of the optical    axis;-   h3.2) Transforming the image based on the new estimated direction of    the optical axis.

Preferably, steps h1) through h6) as well as i) and j) and preferablythe steps h3.1) and h3.2) are repeated continuously with changes in theoperator command.

In the event of the entry of an operator command to change the directionof movement of the vehicle, the extrapolation of the previous image maypreferably include the following:

-   k1) Determining the direction of the optical axis of the image    acquisition device from the vehicle state data transmitted in step    d);-   k2) Determining the control command, that is, the commanded    direction of movement or the commanded movement target from the    entered operator command;-   k3) Determining a new image by changing the previous image while    taking into account the control command determined in step k2);-   k4) Transferring the control command determined in step k2) to the    vehicle;-   k5) Calculating new vehicle control commands (for example,    acceleration commands) in the onboard computer of the vehicle using    the control commands determined in step k2) as well as the navigated    vehicle data (such as, for example, vehicle position, vehicle    location and direction of movement) and-   k6) Carrying out the vehicle control command determined in step k5).

It can also be particularly advantageous if the following steps arecarried out to determine the new image in the above-noted step k3):

-   k3.1) Estimating a new vehicle location and direction of movement    starting from the previous vehicle location and direction of    movement based on a model of the vehicle dynamics;-   k3.2) Transforming the image based on the new estimated vehicle    location and direction of movement.

Preferably, steps k1) through k6) as well as i) and j) and preferablysteps k3.1) and k3.2) are continuously repeated with changes in theoperator command.

It is also advantageous if the optical axis of the image acquisitiondevice is adjusted in the direction of the target position.

Embodiments of the invention are also directed to a control system thatincludes at least one steearable vehicle, at least one operating unitfor the vehicle and a data transfer device for the exchange of dataand/or signals between the vehicle and the operating unit. The vehiclehas at least one seeker head provided with an image acquisition device,an image signal processing device, a vehicle-side transmitter andreceiver unit as well as a control device for the vehicle. The operatingunit has an operating unit-side transmitter and receiver unit, a signalprocessing device, an image display device and a command entry device. Adata compression device is assigned to the vehicle-side transmitter andreceiver unit and a data compression device is assigned to the operatingunit-side transmitter and receiver unit, and the signal processingdevice of the operating unit is embodied such that it transfers imagedata of consecutive images received by the image acquisition device ofthe vehicle to the image display device, wherein after the entry of anoperator command to change the direction of movement of the vehicleand/or to change the direction of view of the image acquisition deviceinto the operating unit, a new image is calculated by extrapolation ofthe last received and displayed image according to a method according tothe invention according to one of the preceding claims taking intoaccount vehicle state data and the entered operator command and isdisplayed on the image display device. The control command determinedfrom the operator command is then transmitted to the vehicle by the datatransfer device and converted in the vehicle.

A control system of this type, which operates according to theembodiments of the method, represents a cost-effective remote control ofthe vehicle by a human operator from an operating station having theoperating device.

Preferably, the vehicle is an aircraft, such as, for example, an armedmissile, which is remotely controlled from a ground station.

The advantages of remote control according to the “man-in-the-loop”method, listed above, are realized in this control system. Consequently,the operator can orient himself by pivoting the image acquisition deviceduring the drive or during the flight. Furthermore, when approaching atarget, the operator can carry out path corrections based on the imageshown on the image display device or even abort a mission.

When the vehicle is armed, preferably is formed by an armed missile, thecontrol system according to the invention forms a new type of weaponsystem.

Embodiments of the invention are directed to a method for providingcontrol commands for a vehicle steerable by an operating unit or for animage acquisition device of the vehicle by sequential display ofacquired image data on an image display device of the operating unitconnected to the vehicle via a data transfer connection. The methodincludes acquiring data of an image recorded by the image acquisitiondevice, acquiring vehicle state data at a time the image was recorded,and compressing the image data. The method also includes transferringthe compressed image data and the vehicle state data via the datatransfer connection to the operating unit, decompressing the transferredimage data to form a displayable image, and displaying the displayableimage on the image display device. The method further includes enteringan operator command into the operating unit to at least one of change adirection of movement of the vehicle and change a direction of view ofthe image acquisition device, calculating a new image from thedisplayable image that takes into account the transferred vehicle statedata and the entered operator command, displaying the new image on theimage display device instead of the displayable image, and transferringa control command from the operator command to the vehicle via the datatransfer connection.

According to embodiments of the instant invention, the acquiring of dataof an image, the acquiring of vehicle state data, the compressing of theimage data, the transferring of the compressed image data, thedecompressing of the transferred image data, and the displaying of thedisplayable image on the image display device may be repeated.

In accordance with other embodiments, the new image can be calculated byextrapolation from a previous displayable image.

Moreover, the vehicle state data can, include data related to anavigated position of the vehicle, a location at least of thelongitudinal axis of the vehicle and its alignment, and a targetposition to be targeted by the vehicle. Also, the vehicle state data mayinclude data related to a direction of an optical axis of the imageacquisition device. At least one of: a direction to the target position,the direction of the optical axis, and a direction of movement of thevehicle can be displayed in the image. Further, the at least one of: thedirection to the target position, the direction of the optical axis, andthe direction of movement of the vehicle may be displayed in the imageas cross hairs.

According to further embodiments of the invention, when an operatorcommand to change the viewing angle of the image acquisition device isentered, the extrapolation of the previous image can include determiningthe direction of the optical axis of the image acquisition device fromthe vehicle state data transmitted, determining the commanded directionof the optical axis from the entered operator command, determining a newimage by changing the previous image taking into account the commandeddirection of the optical axis, transmitting the commanded direction ofthe optical axis to the vehicle, calculating new commanded gimbal anglesfor the image acquisition device in the onboard computer of the vehicleusing the transmitted commanded direction of the optical axis and thenavigated vehicle location, and pivoting the image acquisition deviceaccording to the calculated new commanded gimbal angles. Further, thedetermining of the new image can include estimating a new direction ofthe optical axis starting from the previous direction of the opticalaxis based on a model of the vehicle dynamics and the gimbal dynamics ofthe pivotable image acquisition device using the commanded direction ofthe optical axis, and transforming the image based on the new estimateddirection of the optical axis.

According to further embodiments, the determining the direction of theoptical axis of the image acquisition device; the determining of thecommanded direction of the optical axis; the determining of a new image;the transmitting of the commanded direction; calculating new commandedgimbal angles; the pivoting the image acquisition device; the displayingof the new image; and the transferring of a control command may berepeated continuously with changes in the operator command. Further, theestimating of a new direction of the optical axis; and the transformingof the image can also be repeated continuously with changes in theoperator command.

In accordance with other embodiments of the invention, with the entry ofan operator command to change the direction of movement of the vehicle,the extrapolation of the previous image may include determining thedirection of the optical axis of the image acquisition device from thetransmitted vehicle state data, determining the control command that isone of a commanded direction of movement or a commanded movement targetfrom the entered operator command, determining a new image by changingthe previous image while taking into account the control command,transferring the control command to the vehicle, calculating new vehiclecontrol commands in the onboard computer of the vehicle using thecontrol commands as well as the navigated vehicle data, and carrying outthe calculated vehicle control command. Further, the determining of thenew image may include estimating a new vehicle location and direction ofmovement starting from the previous vehicle location and direction ofmovement based on a model of the vehicle dynamics, and transforming theimage based on the new estimated vehicle location and direction ofmovement.

According to further aspects of the method, the determining of thedirection of the optical axis of the image acquisition device; thedetermining of the control command; the determining of a new image; thetransferring of the control command to the vehicle; the calculating ofnew vehicle control commands in the onboard computer of the vehicle; thecarrying out of the calculated vehicle control command; the displayingof the new image; and the transferring of a control command can berepeated continuously with changes in the operator command. Moreover,the estimating of a new vehicle location and direction of movement; andthe transforming of the image can also be continuously repeated withchanges in the operator command.

In accordance with still yet other embodiments of the present invention,the method can further include setting the direction of the optical axisof the image acquisition device in the direction of the target position.

Embodiments of the invention are directed to a control system thatincludes at least one steerable vehicle having at least one seeker headprovided with an image acquisition device, an image signal processingdevice, a vehicle-side transmitter and receiver unit, and a controldevice for the vehicle. A data compression device is assigned to thevehicle-side transmitter and receiver unit. The control system alsoincludes at least one operating unit for the vehicle having an operatingunit-side transmitter and receiver unit, a signal processing device, animage display device and a command entry device, and a data compressiondevice is assigned to the operating unit-side transmitter and receiverunit. A data transfer device is provided for the exchange of at leastone of data and signals between the vehicle and the operating unit. Thesignal processing device of the operating unit is structured andarranged to transfer image data of consecutive images received by theimage acquisition device of the vehicle to the image display device, thecommand entry device is structured and arranged to receive entry of anoperator command to at least one of change a direction of movement ofthe vehicle and change a direction of view of the image acquisitiondevice into the operating unit, and, in response to the received entry,a new image that is calculated by extrapolation of a last received anddisplayed image taking into account vehicle state data and the enteredoperator command is displayed on the image display device, and theoperator side transmitter is structured and arranged to transmit acontrol command determined from the operator command to the vehicle sidereceiver via the data transfer device, and vehicle includes a converterto convert the received control command.

Other exemplary embodiments and advantages of the present invention maybe ascertained by reviewing the present disclosure and the accompanyingdrawing.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is further described in the detailed descriptionwhich follows, in reference to the noted plurality of drawings by way ofnon-limiting examples of exemplary embodiments of the present invention,in which like reference numerals represent similar parts throughout theseveral views of the drawings, and wherein

FIG. 1 diagrammatically illustrates a control system according to theinvention with the operational sequence of the method according to theinvention during the pivoting of the image acquisition device using theexample of a missile;

FIGS. 2A and 2B illustrates an image prediction produced by the methodaccording to the invention;

FIG. 3 diagrammatically illustrates a control system according to theinvention with the operational sequence of the method according to theinvention with a change of the path of the vehicle using the example ofa missile; and

FIGS. 4A and 4B illustrate an image display before and after a pivotcommand has been issued.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

The particulars shown herein are by way of example and for purposes ofillustrative discussion of the embodiments of the present invention onlyand are presented in the cause of providing what is believed to be themost useful and readily understood description of the principles andconceptual aspects of the present invention. In this regard, no attemptis made to show structural details of the present invention in moredetail than is necessary for the fundamental understanding of thepresent invention, the description taken with the drawings makingapparent to those skilled in the art how the several forms of thepresent invention may be embodied in practice.

FIG. 1 shows a diagrammatic concept rendering of a control systemaccording to the invention using the example of an armed missile. Thecontrol system includes a missile 2 and an operating unit 1 for theremote control of the missile 2. FIG. 1 shows the signal flow and theoperational sequence that takes place with the entry of a pivot commandfor the image acquisition device according to the embodiments of themethod.

The operating unit 1 is connected to the missile 2 via a datacommunication connection 3, referred to as a data link or also as a datatransfer connection, for the reciprocal exchange of data and/or signals.

The data communication connection 3 comprises a first transmitter andreceiver device 31 assigned to the operating unit 1, a secondtransmitter and receiver device 32 assigned to the missile 2 and awireless transmission link 30 for the data and/or signals.

An operator designated as operator 4 operates the operating station 2,which contains, among other things, operating elements of a commandentry device 10, e.g., an entry instrument (such as, a control stick orjoystick) and to which an image display device 12 is assigned.

The missile 2 has an image acquisition device 20, which contains arecording sensor, e.g., a camera, for recording images, as well as acomputer for image processing and for image data compression. The imageacquisition device 20 or the camera thereof is arranged in a pivotablemanner in the nose of the missile 2 in a known manner.

Furthermore, the missile 2 is provided with a navigation device, whichis a constituent of a device 22 for determining missile state data. Thisdevice 22 for determining missile state data contains suitable sensorsfor determining at least the missile location and the flight direction,and can deter mine the current position of the missile via thenavigation device. The missile 2 is furthermore provided with an onboardcomputer—not shown—which is able to perform calculations to control themissile and to control the image acquisition device.

The image acquisition device 20 of the missile 2 records an image. Thisimage is processed either in the computer of the image acquisitiondevice 20 or in the onboard computer and, if necessary optimized, e.g.,by a contrast correction. Subsequently, the data of this processed imageare compressed for the data transfer.

Parallel to recording the image, an acquisition of missile state data iscarried out at the time the image is recorded. These missile state datacomprise at least the navigated position of the missile 2 and its flightposition and air traffic control, the target position and, with apivotable recording sensor of the image acquisition device, also thegimbal angle measured on the pivot axes, i.e., the gradients of theoptical axis of the recording sensor of the image acquisition device 20relative to the missile-fixed axes. The information gained herefrom onthe direction and the position of the optical axis of the image recordedis encrypted together with the compressed image data and transmitted asan encoded data stream from the second transmitter/receiver device 32via the wireless transmission link 30 to the first transmitter/receiverdevice 31 and from there to the operating station 1.

In the operating station 1, the arriving encoded data are first decodedand the direction and the position of the optical axis of the image aredetermined. The image data contained in the data received aresubsequently decompressed and shown to the operator 4 on the imagedisplay device 12. Preferably, additional information is inserted intothe image shown such as, for example, the direction of the optical axisin the form of cross hairs. The operability and thus the controllabilityof the missile or the pivotable image acquisition device 20 arefacilitated thereby.

When the operator 4 actuates the joystick of the operating elements 10for the purpose of pivoting the image acquisition device 20 or therecording sensor of the image acquisition device, the processesdescribed below take place in the operating unit.

First the image transmitted from the missile is shown on the imagedisplay device 12 of the operating unit 1. The direction of the opticalaxis of the transmitted image is determined from the missile state datareceived. Due to the latency in the data transfer from the missile 2 viathe data transfer connection 3 to the operating unit 1, the image shownand the determined optical axis do not correspond to the real-timestate, but to a state in the past.

If the operator 4 now gives a command via the joystick to pivot theoptical axis of the image acquisition device or the recording sensorthereof, it would again take a long time until the image resulting fromthe pivot motion is displayed on the image display device. This latencyleads to a high time lag in the control loop between the operator 4, thecontrol and the movement of the image acquisition device 20.

To compensate for this latency, on the basis of the operating dataentered via the joystick of the operating elements 10, the operatingunit determines the direction of the commanded optical axis and thedifference from the optical axis of the image currently shown by theimage display device 12. A model of the missile dynamics and the gimbaldynamics, i.e., the dynamics of the swivel motions of the imageacquisition device or the recording sensor thereof, is implemented inthe operating unit 1. Based on this movement model, on the basis of thereal optical axis of the displayed image and the calculated commandedoptical axis, the optical axis at the current time is estimated in acomputer of the operating unit and the most current image received fromthe missile 2 is transformed according to this estimated real opticalaxis. This so-called image prediction provides a predicted (estimated)image which is then displayed by the image display device 12 instead ofthe last transmitted image.

In the simplest case, this image prediction can be carried out bydisplacing and if necessary scaling the last image received. The imageparts missing thereby which were not visible in the original image aresupplemented in a suitable manner, e.g., by single-color bars or by agrid structure, as is shown in the exemplary representation of FIGS. 2Aand 2B.

FIG. 2A shows a received image and FIG. 2B shows a predicted image,which has been produced by displacing the received image downwards tothe right. The region of the predicted image for which no real imageinformation is available, was replaced by a grid structure in therepresentation of FIG. 2B.

The retrieval of new operating commands and the predictionrepresentation of the respectively predicted image are constantlyrepeated. Thus, the operator receives a direct optical feedback on hisuser interventions, that is, on his operator commands entered via theoperating elements 10. Control commands in the form of the data for thecommanded optical axis are determined from these operator commands oroperating commands and transmitted via the data transfer connection 3 tothe missile 2.

The missile first determines its own missile location and missileposition by its navigation device and calculates from this navigatedmissile location and from the received commanded optical axis the newcommanded gimbal angles for the pivotable image acquisition device 20 orfor the pivotable recording sensor thereof. Subsequently, a pivoting ofthe image acquisition device 20 or of the recording sensor thereof iscarried out on the basis of these new commanded gimbal angles.

The essential factor of this principle according to the invention is theasynchronous realization which mitigates the latency problem. As soon asa new transmitted image is available, it replaces the predicted image.Since the newly received image is similar to the previously displayedpredicted image, the optically discernible image jump will be slight andthe operator is not impeded in his operating activity. Ideally, only theimage parts missing in the predicted image are then replaced.Analogously, a new image can be recorded in the missile, even while thelast operator command has not yet been completely transmitted to themissile and realized there. To this end, it is necessary to alwaystransmit with the image the measured gimbal angle (i.e., the gimbalangle present at the moment the image was recorded) or, using thenavigated missile location, the current direction of the optical axiscalculated therefrom.

Based on FIG. 3, an example is given below in which the operatorcommands entered by the operator 4 via the operating elements 10 areused to change the flight path, when the missile is to approach a newtarget, for example, or when the flight direction to the target is to bechanged. The procedure and the sequence are essentially the same as withthe pivoting of the optical axis of the image acquisition device 20, ashas been described in connection with FIG. 1.

Here, too, there can be a marked latency between the image transmittedand displayed by the image display device 12 and the target informationavailable in the missile at the time of display.

In addition to the data of the optical axis of the recording sensor orthe image acquisition device 20, with this form of the realization ofthe method according to the invention, the commanded direction to thetarget determined by the operator command, which the operator 4 entersinto the operating unit 1 via the operating elements 10, is transmitted.The direction of the optical axis and the direction to the target aredisplayed, e.g., as cross hairs on the display device 12 of theoperating unit I. The operator 4 can influence both directions by theoperating elements 10. If the target is not located in the direction ofthe optical axis, that is, not in the center of the image, the targetposition will jump from image to image and ultimately shift out of theimage. This makes the correction of the target position by the operatormuch more difficult. It is therefore expedient during the final approachto regulate the optical axis of the seeker head in the direction of theplanned target position.

The operator command for the target position entered by the user 4 viathe operating elements 10 is first used in the operating unit 1 forcalculating a commanded target position. This commanded target positiontogether with the current target position received from the missile 2 isthen subjected to the image prediction calculation on the basis of themodel of the missile dynamics and the gimbal dynamics. The predictedimage obtained is then shown on the image display device 12, as in theexample of FIG. 1. At the same time, the data of the commanded targetposition, i.e., the commanded direction from the missile 2 to thetarget, are transmitted via the data transfer connection 3 to themissile 2. There the commanded target position is calculated in thedevice 22 to determine missile state data based on the transmittedcommanded direction from the missile to the target position and used tocalculate the position of the target in the recorded image. These imagedata containing the position of the target are then transmitted to theoperating unit as in the example of FIG. 1.

FIGS. 4A and 4B show a prediction of the representation of a changingtarget position on the basis of a real image (FIG. 4A) recorded by theimage acquisition device 20 of the missile 2 and a predicted image (FIG.4B).

In FIG. 4A, the direction of the optical axis of the image acquisitiondevice 20 (center of the image) is marked by first cross hairs A and theflight direction of the missile, which corresponds to the velocityvector of the missile, is marked by second cross hairs B. The opticalaxis of the image acquisition device 20 is directed at the target to beapproached and, in this example, consequently corresponds to the targetvector directed from the missile 2 to the target. The velocity vector ofthe missile (cross hairs B) deviates during the flight due to theselected flight path (for example, a ballistic path) from the opticalaxis or the target vector. Towards the end of the flight, the crosshairs B will also move into the center of the image.

FIG. 4B furthermore shows by third cross hairs C the direction to a newtarget aimed at by the operator and selected via the actuation of theoperating unit. In the case of the missile simulated here, the seekerhead equipped with the image acquisition device 20 (and with itultimately the entire missile) pivots in the direction of the new targetaimed at.

If the image is pivoted for a new target instruction (FIG. 4B), thecross hairs A, which represented the previous optical axis and thereforein FIG. 4A are congruent with the cross hairs C, shift from the centerof the image of the predicted image. After some time, the seeker headwill actually carry out the pre-simulated swivel motion and theassociated cross hairs A will again move into the center of the image.

Ideally, (FIG. 4A) cross hairs A and cross hairs C are thereforecongruent and towards the end of the flight cross hairs B will also moveinto the center of the image.

Even though the invention has been described based on a missile as avehicle in the exemplary embodiments, the method according to theinvention and the control device according to the invention can also berealized with other vehicles, such as, for example, ground vehicles,watercraft or spacecraft.

Reference numbers in the claims, the specification and the drawings areused only for the better understanding of the invention and are notintended to restrict the scope of protection.

It is noted that the foregoing examples have been provided merely forthe purpose of explanation and are in no way to be construed as limitingof the present invention. While the present invention has been describedwith reference to an exemplary embodiment, it is understood that thewords which have been used herein are words of description andillustration, rather than words of limitation. Changes may be made,within the purview of the appended claims, as presently stated and asamended, without departing from the scope and spirit of the presentinvention in its aspects. Although the present invention has beendescribed herein with reference to particular means, materials andembodiments, the present invention is not intended to be limited to theparticulars disclosed herein; rather, the present invention extends toall functionally equivalent structures, methods and uses, such as arewithin the scope of the appended claims.

LIST OF REFERENCE NUMBERS

-   1 Operating unit-   2 Missile (vehicle)-   3 Data transfer connection-   4 Operating person or operator-   10 Command entry device-   12 Image display device-   20 Image acquisition device-   22 Device for determining missile state data-   30 Transmission link-   31 First transmitter and receiver device-   32 Second transmitter and receiver device

1. A method for providing control commands for a vehicle steerable by anoperating unit or for an image acquisition device of the vehicle bysequential display of acquired image data on an image display device ofthe operating unit connected to the vehicle via a data transferconnection, the method comprising: acquiring data of an image recordedby the image acquisition device; acquiring vehicle state data at a timethe image was recorded; compressing the image data; transferring thecompressed image data and the vehicle state data via the data transferconnection to the operating unit; decompressing the transferred imagedata to form a displayable image; displaying the displayable image onthe image display device; entering an operator command into theoperating unit to at least one of change a direction of movement of thevehicle and change a direction of view of the image acquisition device,calculating a new image from the displayable image that takes intoaccount the transferred vehicle state data and the entered operatorcommand; displaying the new image on the image display device instead ofthe displayable image; and transferring a control command from theoperator command to the vehicle via the data transfer connection.
 2. Themethod in accordance with claim 1, wherein the acquiring of data of animage, the acquiring of vehicle state data, the compressing of the imagedata, the transferring of the compressed image data, the decompressingof the transferred image data, and the displaying of the displayableimage on the image display device are repeated.
 3. The method inaccordance with claim 1, wherein the new image is calculated byextrapolation from a previous displayable image.
 4. The method inaccordance with claim 1, wherein the vehicle state data comprise datarelated to a navigated position of the vehicle, a location at least ofthe longitudinal axis of the vehicle and its alignment, and a targetposition to be targeted by the vehicle.
 5. The method in accordance withclaim 4, wherein the vehicle state data comprise data related to adirection of an optical axis of the image acquisition device.
 6. Themethod in accordance with claim 5, wherein at least one of a directionto the target position, the direction of the optical axis, and adirection of movement of the vehicle is displayed in the image.
 7. Themethod in accordance with claim 6, wherein the at least one of thedirection to the target position, the direction of the optical axis, andthe direction of movement of the vehicle is displayed in the image ascross hairs.
 8. The method in accordance with claim 3, wherein, when anoperator command to change the viewing angle of the image acquisitiondevice is entered, the extrapolation of the previous image comprises:determining the direction of the optical axis of the image acquisitiondevice from the vehicle state data transmitted; determining thecommanded direction of the optical axis from the entered operatorcommand; determining a new image by changing the previous image takinginto account the commanded direction of the optical axis; transmittingthe commanded direction of the optical axis to the vehicle; calculatingnew commanded gimbal angles for the image acquisition device in theonboard computer of the vehicle using the transmitted commandeddirection of the optical axis and the navigated vehicle location; andpivoting the image acquisition device according to the calculated newcommanded gimbal angles.
 9. The method in accordance with claim 8,wherein the determining of the new image comprises: estimating a newdirection of the optical axis starting from the previous direction ofthe optical axis based on a model of the vehicle dynamics and the gimbaldynamics of the pivotable image acquisition device using the commandeddirection of the optical axis; and transforming the image based on thenew estimated direction of the optical axis.
 10. The method inaccordance with claim 8, wherein the determining the direction of theoptical axis of the image acquisition device; the determining of thecommanded direction of the optical axis; the determining of a new image;the transmitting of the commanded direction; calculating new commandedgimbal angles; the pivoting the image acquisition device; the displayingof the new image; and the transferring of a control command are repeatedcontinuously with changes in the operator command.
 11. The method inaccordance with claim 10, wherein the estimating of a new direction ofthe optical axis; and the transforming of the image are repeatedcontinuously with changes in the operator command.
 12. The method inaccordance with claim 1, wherein, with the entry of an operator commandto change the direction of movement of the vehicle, the extrapolation ofthe previous image comprises: determining the direction of the opticalaxis of the image acquisition device from the transmitted vehicle statedata; determining the control command that is one of a commandeddirection of movement or a commanded movement target from the enteredoperator command; determining a new image by changing the previous imagewhile taking into account the control command; transferring the controlcommand to the vehicle; calculating new vehicle control commands in theonboard computer of the vehicle using the control commands as well asthe navigated vehicle data; and carrying out the calculated vehiclecontrol command.
 13. The method in accordance with claim 12, wherein thedetermining of the new image comprises: estimating a new vehiclelocation and direction of movement starting from the previous vehiclelocation and direction of movement based on a model of the vehicledynamics; and transforming the image based on the new estimated vehiclelocation and direction of movement.
 14. The method in accordance withclaim 12, wherein the determining of the direction of the optical axisof the image acquisition device; the determining of the control command;the determining of a new image; the transferring of the control commandto the vehicle; the calculating of new vehicle control commands in theonboard computer of the vehicle; the carrying out of the calculatedvehicle control command; the displaying of the new image; and thetransferring of a control command are repeated continuously with changesin the operator command.
 15. The method in accordance with claim 14,wherein the estimating of a new vehicle location and direction ofmovement; and the transforming of the image are continuously repeatedwith changes in the operator command.
 16. The method in accordance withclaim 1, further comprising setting the direction of the optical axis ofthe image acquisition device in the direction of the target position.17. A control system comprising: at least one steerable vehicle havingat least one seeker head provided with an image acquisition device, animage signal processing device, a vehicle-side transmitter and receiverunit, and a control device for the vehicle, wherein a data compressiondevice is assigned to the vehicle-side transmitter and receiver unit; atleast one operating unit for the vehicle having an operating unit-sidetransmitter and receiver unit, a signal processing device, an imagedisplay device and a command entry device, wherein a data compressiondevice is assigned to the operating unit-side transmitter and receiverunit; and a data transfer device for the exchange of at least one ofdata and signals between the vehicle and the operating unit, wherein thesignal processing device of the operating unit is structured andarranged to transfer image data of consecutive images received by theimage acquisition device of the vehicle to the image display device,wherein the command entry device is structured and arranged to receiveentry of an operator command to at least one of change a direction ofmovement of the vehicle and change a direction of view of the imageacquisition device into the operating unit, and, in response to thereceived entry, a new image that is calculated by extrapolation of alast received and displayed image taking into account vehicle state dataand the entered operator command is displayed on the image displaydevice, and wherein the operator side transmitter is structured andarranged to transmit a control command determined from the operatorcommand to the vehicle side receiver via the data transfer device, andvehicle includes a converter to convert the received control command.